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Technical  Report  on  DOD/Naval  Undersea  Warfare  Center  grant 
N66604-09-M-4141: 

Object  Detection  in  Naval  Environments  using  Information-theoretic  Approaches 


Abstract  We  have  investigated  image  analysis  methods  for  tracking  moving  ob¬ 
jects  in  marine  environments.  We  focused  on  information-theoretic  approaches  that 
compare  image  regions  of  interest  with  the  statistical  models  of  the  objects  to  be 
tracked.  We  evaluated  tracking  performance  of  the  mean-shift  and  block-matching 
algorithms  on  videos  of  waterways.  It  is  challenging  to  track  naval  vessels  in  these 
videos  due  to  glare  from  the  water  surface,  up  and  down  movements  of  vessels,  and 
occlusions.  We  found  that  the  performance  of  the  mean-shift  tracking  approach  was 
superior  to  the  performance  of  the  block-matching  approach  in  videos  with  low  con¬ 
trast  between  the  naval  vessels  to  be  tracked  and  the  surrounding  water  surface  and 
where  the  scale  and  orientation  of  the  vessels  changed. 


1  Introduction 

Real-time  large-area  surveillance  is  increasingly  performed  in  naval  environments 
by  unmanned  vehicles  (UxV)  equipped  with  visual  sensors.  Imagery  inputs  are 
radio-transmitted  to  operators  for  analysis.  Typical  operator  display  devices  report 
objects  of  interest  at  ranges  at  which  the  operator  cannot  recognize  the  type  of  ob¬ 
ject,  i.e.,  a  fixed  installation,  or  a  moving  commercial  fishing  ship  or  naval  vessel. 
The  operator  must  decide  whether  or  not  to  further  investigate  each  detected  object. 
Further  investigation  exposes  the  UxV  to  risks  and  is  expensive,  because  it  involves 
maintaining  a  continuous  radio  link,  moving  the  UxV  in  potentially  rough  seas,  and 
monitoring  its  reported  imagery  continuously  until  the  object  becomes  recognizable. 
The  goal  of  this  research  project  was  to  investigate  video-analysis  approaches  that 
can  speed  up  and  automate  the  recognition  process.  As  a  first  step  towards  this  goal, 
we  identified  techniques  for  tracking  moving  objects  in  marine  environments.  We 
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focused  on  information-theoretic  approaches  that  compare  image  regions  of  inter¬ 
est  with  the  statistical  models  of  the  objects  to  be  track,ed  In  particular,  we  applied 
the  mean-shift  algorithm  to  the  problem  of  tracking  histogram-based  target  repre¬ 
sentations  as  proposed  by  Comaniciu,  Ramesh,  and  Meer  (2003)  and  Bibby  and 
Reid  (2005),  where  the  objects  are  represented  by  color  histograms.  We  also  imple¬ 
mented  a  block  matching  approach  by  Gyaourova,  Kamath,  and  Cheung  (2003)  and 
compared  the  performance  of  the  two  methods. 


2  Methods 

2.1  The  Mean  Shift  Algorithm 

The  mean  shift  algorithm  is  implemented  by  the  comparison  of  color  histograms. 
The  histogram  p  of  the  object  to  be  tracked  is  obtained  from  the  reference  frame 
and  is  compared  to  the  histogram  <7  in  a  subimage  of  a  subsequent  frame.  We  used 
the  Bhattacharyya  coefficient  as  a  measure  of  similarity  between  the  two  histograms. 
We  implemented  a  linearized  version  of  the  coefficient  that  uses  a  kernel  that  ensures 
that  pixels  near  the  center  of  the  subimage  are  weighted  more  than  pixels  near  the 
edge  of  the  subimage  (Bibby  and  Reid,  2005). 


2.2  The  Block  Matching  Algorithm 

In  the  block  matching  algorithm,  images  are  segmented  into  a  number  of  regions 
or  “blocks.”  Each  block  in  the  current  image  frame  is  matched  with  a  block  in  the 
destination  frame  by  shifting  it  over  a  predefined  neighborhood  of  pixels  in  the  des¬ 
tination  frame.  The  blocks  are  compared  with  a  similarity  measure  that  computes 
the  mean  absolute  difference  (MAD)  of  corresponding  brightness  values  in  the  two 
blocks  (Gyaourova,  Kamath,  and  Cheung,  2003).  The  search  process  over  the  pre¬ 
defined  neighborhood  is  performed  using  a  “two  dimensional  logarithmic  search” 
(TDL)  algorithm  (Gyaourova,  Kamath,  and  Cheung,  2003).  The  algorithm  is  de¬ 
scribed  as  follows: 

1.  Select  an  initial  step  size  s. 

2.  Compare  the  current  template  block  with  the  block  at  the  center  of  the  destination 
image  and  blocks  at  distance  s  above,  below,  left,  and  right  of  the  center.  Compute 
the  similarity  measure  for  the  five  block  comparisons. 

3.  If  the  block  at  the  center  is  most  similar  among  the  five,  halve  the  step  size: 
s  *r—  s/2,  If  one  of  the  other  four  blocks  is  most  similar,  select  this  block  as  a  new 
center  in  the  destination  image  and  repeat  step  2. 

4.  When  the  step  size  s  becomes  1 ,  compare  the  template  block  to  all  the  nine  blocks 
in  the  destination  image  around  the  currently  chosen  center  block.  The  block 
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among  these  nine  that  best  matches  with  the  template  block  is  selected  as  the 
image  region  that  represents  the  tracked  object. 


3  Experimental  Results 

3.1  Experimental  Results  for  the  Mean  Shift  Algorithm 

The  white  boat  shown  in  Figure  1  has  a  sufficient  level  of  intensity  contrast  to  the 
water  surface  and  can  be  tracked  reliably  with  the  mean-shift  tracker.  The  darker 
row  boat  in  the  same  video  sequence  is  also  tracked  successfully  with  the  mean- 
shift  tracker  (Figure  2).  Note  however  that  loss  of  track  occurs  when  the  row  boat 
entered  a  region  of  the  body  of  water  where  the  reflection  of  a  building  created  a 
background  region  with  a  similar  intensity  distribution. 


Fig.  1  Tracking  a  bright  power  boat  with  the  mean-shift  tracker. 


An  advantage  of  the  mean-shift  tracker  is  that  it  can  handle  frame-to-frame 
changes  in  scale  of  the  object  (Figure  3)  and  orientation. 

The  mean-shift  tracker  did  not  perform  well  when  the  illumination  of  the  body  of 
water  changed  (Figures  4  and  5).  In  our  examples,  loss  of  track  occurred  in  frame  50 
when  we  simulated  an  illumination  change  in  frame  26. 
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Frame  =  1 


Frame  =  150 


Fig.  2  Tracking  a  dark  row  boat  with  the  mean-shift  tracker. 


Fig.  3  Tracking  a  boat  that  is  moving  towards  the  camera  with  the  mean-shift  tracker. 


3.2  Experimental  Results  for  the  Block  Matching  Algorithm 

We  tested  the  block-matching  algorithm  on  the  same  videos  as  the  mean-shift  al¬ 
gorithm.  The  white  boat  shown  in  Figure  6  can  be  tracked  reliably  with  the  block¬ 
matching  tracker.  The  darker  row  boat  in  the  same  video  sequence  is  also  tracked 
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Fig.  4  Tracking  a  moving  boat  with  the  mean-shift  tracker  in  an  evironment  with  an  increased 
level  of  brightness. 


successfully  with  the  block-matching  tracker  (Figure  7),  but,  as  was  the  case  for  the 
mean-shift  tracker,  loss  of  track  occurs  when  the  row  boat  entered  a  region  of  the 
body  of  water  with  the  building  reflection. 

The  block-matching  algorithm  could  not  detect  and  track  changes  in  object  scale 
or  orientation. 


4  Conclusions  and  Future  Work 

For  tracking  of  vessels  moving  in  a  marine  environment,  we  found  that  the  per¬ 
formance  of  the  mean-shift  tracker  was  superior  to  the  performance  of  the  block¬ 
matching  tracker  in  videos  with  low  contrast  between  the  vessels  and  the  surround¬ 
ing  water  surface  and  where  the  scale  and  orientation  of  the  vessels  changed. 

Future  work  includes  building  motion  models  of  naval  vessels  that  can  be  used 
to  support  tracking  the  vessel  and  recognition  of  the  type  of  vessel.  It  also  includes 
developing  and  testing  approaches  to  cope  with  occlusions  of  tracked  vessels. 
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Fig.  5  Tracking  a  moving  boat  with  the  mean-shift  tracker  in  an  evironment  with  a  decreased  level 
of  brightness. 
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Fig.  6  Tracking  a  bright  power  boat  with  the  block-matching  tracker. 


Frame  =  1 


Frame  =  150 


Frame  =  300 


Fig.  7  Tracking  a  dark  row  boat  with  the  block-matching  tracker. 


